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Abstract 

Language testing is a relatively new field of interest that has become of common interest in the last twenty years not only for 
traditional purposes but because of the power that language testing has acquired in relation to immigration and also for its 
implications for educational reforms. The purpose of this paper is to provide a theoretical approach to how audiovisual clues help 
to contextualize the semi-direct interviews in the Spanish University Entrance Examination (P.A.U.) and how contextualization 
helps to trigger and improve the student’s performance. The study suggests the importance of contextualization of speaking tasks 
and how computers can enhance language testing. 
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1. Introduction 

Language testing is a relatively new field of interest that has become of common interest in the last twenty years 
not only for traditional purposes but because of the power that language testing has acquired in relation to 
immigration and also for its implications for educational reforms. Language testing has revealed three relevant 
aspects of the evaluation and assessment process: a) the importance of validation; b) the importance of technology in 
test design and delivery; and c) alternative means of language testing. Computer assisted language testing is now 
accepted as a revolting engine in language assessment that has presented significant evolutions in all these three 
aspects. One of the tasks that has become increasingly integrated in computer assisted language testing is speaking. 
Speaking, relatively expensive and problematic in its delivery until now has found in computers and the Internet a 
valid ground to lower delivery costs and maximize the student’s performance. Computer based speaking tasks or full 
tests allow to integrate audiovisual repertoires that facilitate the student’s integration in the test and trigger the 
testee’s production. 
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The purpose of this paper is to provide a theoretical approach to how audiovisual clues help to contextualize the 
semi-direct interviews in the Spanish University Entrance Examination (P.A.U.) and how contextualization helps to 
trigger and improve the student’s performance. 

2. Prototyping questions for a computer based language test 

In 2004, the Universidad Politecnica de Valencia(Spain) started to design a new oral-written computer tool for 
language testing. In 2007, the Camille researchers started a huge project to design and foresee the possible 
implications of the design and implementation of a University Entrance Examination (P.A.U.) . In those days, it was 
assumed that a written online test would imply little if any difficulties. Thus, the challenge was implementing an 
oral section within the test. In order to achieve that goal, a limited number of researchers worked towards the design 
of the type of questions that would be desirable. So first, the researchers considered what was the exam to provide 
evidence of. So then, a set of goals were related to the tasks. It was assumed that the P.A.U. administrators wanted 
the cope with the students’ needs in a foreign language in a non English speaking setting. That means that English in 
that sense should be use to complement lessons or even attend classes in a foreign language but the interaction out of 
class would be for communication purposes rather than to live in an English speaking setting (as opposed to 
TOEFL). In that sense students were to provide evidence of the following aspects: 


Table 1. Practical applications of the intended speaking tasks in the new P.A.U. test. 


Practical application 

Socializing (meeting people 
and asking identification 
questions 

Obtaining information from 
a lecture 

Giving short presentations 


Describing prompts 


Type of oral interaction 

Dialogue 

Giving complete answers to informative 
questions - Dialogue 
Being able to trim a topic of their 
knowledge and/or interest and speak about it 
- Monologue 

Being able to describe a diagram or a 
picture 


Length of the speaker’s turn 

one to ten seconds (very short 
responses) 

Up to one minute 
Between three and five minutes 


About three minutes 


2 . 1 Kinds of inferences 

The principle that underlies the inferences obtained through the test should provide information enough to deduce 
that a student would be able to perform properly in the given context. Thus, inferences should provide that type of 
information. The scores obtained in the test should be valid to obtain generalizations about the testee’s proficiency 
in the foreign language as well as in the expected academic context while facilitating the interpretation for all the 
different test stakes (especially the candidate, the test administrators and possible educational bodies such as the 
Ministry of Education or the prospective university. Therefore, the researchers established a set of expectations 
according the expected inference stage as follows: 
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Table 2. Relationship between the administrators’ expectations and the inference stage. 
Expectations Inference stage 

Descriptors of the students’ proficiency are accurate 

Evaluation 

Oral tasks are appropriate and are valid to measure the students’ 
competence in the test context 

The tasks are valid, reliable and their number is acceptable to measure 
the student’s production. 


Tasks provide enough information and are well contextualized. 


Generalization 


Context does not jeopardize the generalization of the results. 
The results can be related to the writing and reading measures. 


The mental processes required to respond to the oral items fulfill the 
administrators’ principles and objectives. 

The contextualization and place in the test is consistent with the goals 
of the test. 


Analysis and interpretation 


The composition of the items (especially when combining more than 
one language skill) relates with te administrators expectations. 
The results of the test are related to those obtained through another 
test of similar features. 


Extrapolation 


3 Inferences and oral tasks: Semi-direct interviews 

3. 1 Oral tasks in the PA. U. 

Up to today it is difficult to design oral productive tasks in computer assisted language testing. For this research 
project the intention was to integrate four main types of tasks. They all consider watching a mini-clip (that can be 
replaced by an audio recording) before the productive task. The four types are: 

1) Watching and answer questions with short written answers, 

2) Watching and answer multiple choice questions, 

3) Watching and answer questions with long written answers, 

4) Watching and produce mediated oral responses 

The first three can be considered integrated skills tasks because they use different communication channels. 
Naturally, they are not complete in themselves but require of other tasks that cam complement them. The idea of 
integrating tasks is that whenever possible they should include all four language skills. However, although the 
Educational Testing Service tends to use them in thematic clusters (such as attending a class in which the student 
needs to read a diagram or text, listen to a teacher’s speech, give a written response and probably speak by 
answering questions). The research team found that creating banks of such clusters is extremely expensive and not 
many non-profit or institutional organizations can afford it. In conclusion, the team assumed that the same tasks can 
be done through a combination of different items in the same test. In relation to the last type of tasks, the team 
proposed the tasks that corresponded to the expectations from a student as presented in table 1 : 

1) Watching a video clip of a student and answer short informative questions, 

2) Watching a video clip of a teacher teaching a lesson and answer to his questions, 

3) Watching a mini-clip and give a mini-presentation. 

3.2 Using semi-direct interviews 

In working towards the definition of a theoretical framework for the implementation, it was necessary to find a 
construct that assumed all the possible factors to be found in the answering of an video supported oral task. The 


3594 


Jesus Garcia Laborda / Procedia Social and Behavioral Sciences 2 (2010) 3591-3595 


model presented in figure 1 includes a basic diagram of task processing which is repeated as many times as tasks are 
included. 



Figure 1 . A framework of oral task processing and response. 

Oral semi-directed interviews are a set of continuous questions that are interrelated but are not linked. In 
situations such as informal interrelations do not need to follow a particular dialoguing line. For example, if the input 
introduced is “how many sisters do you have?” and the answer is “Six”, there is no need to ask about the “sisters” 
but the following question could be “where do you live”. Although it may be a minor breakdown in a conversation 
pattern, it is still acceptable as a way to gather information in normal conversation and also as a way to obtain the 
respondent’s output. Therefore, semi-direct interviews can be recorded in advance and supported by images giving 
more contextual clues to the testee. 


4. Contextual clues for semi-direct interviews 

4.1 Choosing when to use semi-direct interviews 

Semi-direct interviews are best used when the answers are intended to be short, objective and independent 
questions are necessary (). In the P.A.U. test semi-direct interview are used for informal purposes mostly like 
questions from student to student. But they could also be used to collect responses to a teacher’s lesson. The 
advantage of the informal situation is that when well aimed they can become almost as a whole conversation. 
Current research indicates that they are positive in tests like SOPI and the results obtained at the Universidad 
Politecnica also evidence their positive use to obtain quality production from the test candidates. 

4.2 Contextual clues 

Contextual clues are provided by the use of image and sound to support the candidate’s production. Image and 
sound have proved to be excellent triggers of production especially in short term memory. They also give life and 
realism which in turn also ads a “flow” feeling. Overall, previous studies have shown that although the use of video 
for listening tasks may challenge the task construct (Ockey, 2007), images enrich the students’ responses and very 
productive because they facilitate vocabulary and syntax recognition and use (Verhallen et al., 2006). In the project 
they have been mostly used for oral descriptions and items that require item memorization. They may also seem to 
increase and improve the candidates phonological production (Norrix et al, 2007) since they have a model (as it 
happens in real life in the native-non-native interaction) to follow. 
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4.3 Drawbacks of semi-direct interviews 

The main drawback is their communicative capacity and the lack of connection between questions. Additionally, 
some more talkative candidates may feel frustrated by the lack of argumentative line. According to Mikhailova, 
(2007) semi-directed interviews like those used for the Oral Proficiency Interview (SOPI) may not me as productive 
as thought. However, these students can usually overcome this problem especially more proficient candidates 
(Malabonga et al, (2005). Since they are used to simulate conversations, they lack the guiding line of a regular 
conversation, so it is necessary to use them in situations in which questions can be isolated with no need to develop 
each response. 

5. Conclusions 

Semidirect interviews facilitate the output of students and provide good evidence of competence to warrant the 
students’ proficiency in language testing. They are better used when supported by video clues. Overall, although 
much has been said about the use of video to enrich language learning, little has been said about its effect in 
language testing. As a consequence, it is necessary to devote more studies to analyze its effect in testing. This aspect 
is currently in study in the Universidad Politecnica de Valencia with the hope to observe whether the results support 
the idea that using contextual clues as video for integrated tasks is worth. 

Although no direct funding has been received for this paper, the researcher would like to thank Ministry of 
Education of Spain for supporting the development and implementation of the testing platform hereby mentioned 
under the PAER project (HUM2007-66479-C02-01/FILO) which was used to obtain the theoretical implications 
hereby presented. 
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